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Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )^ Responsive to communication(s) filed on 08/24/2006 . 
2a)[3 This action is FINAL. 2b)D This action is non-final. 
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DETAILED ACTION 

1. This Office Action vacates the finality of the previous Office Action, which was mailed 
on 03/20/2006. In the previous Office Action, Examiner addressed the Applicant's 
amendment filed on 12/23/2005. However, Applicant has submitted a supplemental 
amendment on 03/14/2006. Accordingly, this action is responsive to the Interview 
Summary and Remarks filed 08/24/2006, and a new final rejection is provided below. 

2. Claims 37-63 are currently pending in this application. Claims 1-36 have been canceled. 
Claims 37-63 have been added. Claims 37, 47, 57 and 60 are independent claims. 

It is noted that Applicant states 'new claims 37-64 have been added" (see Remarks, page 
13), however, a review of the Image File Wrapper (IFW) indicates that only claims 37-63 
have been submitted. 

Specification 

3. It is noted that applicant has other related applications (e.g., application # 09/467,23 1 
filed 12/20/1999 and application # 09/785,063 filed 02/16/2001). It is requested that any 
related application be referred to in the first sentence of the specification. Applicant is 



Application/Control Number: 10/733,981 
Art Unit: 2176 



Page 3 



also requested to supply the serial numbers of any other related applications currently 
pending before the U.S Patent & Trademark Office. 

Double Patenting 

4. The nonstatutory double patenting rejection is based on a judicially created doctrine 
grounded in public policy (a policy reflected in the statute) so as to prevent the 
unjustified or improper timewise extension of the "right to exclude" granted by a 
patent and to prevent possible harassment by multiple assignees. See In re Goodman, 
1 1 F.3d 1046, 29 USPQ2d 2010 (Fed. CIT. 1993); In re Longi, 759 F.2d 887, 225 
USPQ 645 (Fed. Cir. 1985); In re van Ornurn, 686 F.2d 937, 214 USPQ 761 (CCPA 
1982); In re Uogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 
418 F.2d 528, 163 USPQ 644 (CCPA 1969). 

A timely filed terminal disclaimer in compliance with 37 C.F.R. 1 1.321(b) would 
overcome an actual or provisional rejection on this ground provided the conflicting 
application or patent is shown to be commonly owned with this application. See 37 
C.F.R. 1 1.78(d). 

Effective January 1, 1994, a registered attorney or agent of record may sign a terminal 
disclaimer. A terminal disclaimer signed by the assignee must fully comply with 37 
CFR 3.73(b). 



5. 



Claims 37-63 are provisionally rejected on the ground of nonstatutory obviousness- 
type double patenting as being unpatentable over claims 83-104 of copending 
Application No. 09/467, 231 (see claims 83-104 as submitted in the amendment filed 
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7/06/2006). Although the conflicting claims are not identical, they are not patentably 
distinct from each other because claim 37 of the instant application and claim 83 of 
copending Application No. 09/467, 231 both claim a data processing apparatus for 
processing media content comprised of a plurality of scenes, the apparatus 
comprising, an input unit operable to input content description data including: a 
plurality of importance attributes each associated with one of the segments and having 
a value representing a degree of contextual importance of the corresponding one of the 
segments; and an output unit operable to output at least one of the segments based on 
at least one of the importance attributes. The difference between claim 37 of the 
instant application and claim 83 of copending Application No. 09/467, 231 is claim 37 
of the instant application further claims a context attribute having value for describing 
a context of the media content and claim 83 of copending Application No. 09/467, 231 
claims the important attributes having a value representing a degree of contextual 
importance of the corresponding one of the plurality of segments. The difference 
would have been obvious to a person of ordinary skill in the art at the time the 
invention was made since the claim limitations appear to have been reworded, 
however, the scope of the invention appears to be generally the same. 



This is a provisional obviousness-type double patenting rejection because the conflicting 
claims have not in fact been patented. 
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6. Claims 37-63 are provisionally rejected on the ground of nonstatutory obviousness- 
type double patenting as being unpatentable over claims 37-49 of copending 
Application No. 09/785, 063 (see claims 37-49 as submitted in the amendment filed 
02/13/2006). Although the conflicting claims are not identical, they are not patentably 
distinct from each other because claim 37 of the instant application and claim 37 of 
copending Application No. 09/785, 063 both claim a data processing apparatus for 
processing media content comprised of a plurality of scenes, the apparatus 
comprising, an input unit operable to input content description data including: a 
context attribute having a value for describing a context of the media content, a 
plurality of importance attributes each associated with one of the segments and having 
a value representing a degree of contextual importance of the corresponding one of the 
segments; and an output unit operable to output at least one of the segments based on 
at least one of the importance attributes. Furthermore, the differences between claim 
37 of the instant application and claim 37 of copending Application No. 09/785, 063 
would have been obvious to a person of ordinary skill in the art at the time the 
invention was made, since claim 37 of the instant application represents the invention 
in broader scope. The claim limitations appear to have been reworded, however, the 
scope of the invention appears to be generally the same. 



This is a provisional obviousness-type double patenting rejection because the conflicting 
claims have not in fact been patented. 
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Claim Rejections - 35 USC §103 



1. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 
1 02 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the 
subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary 
skill in the art to which the subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 

This application currently names joint inventors. In considering patentability of the claims under 35 U.S.C. 103(a), 
the examiner presumes that the subject matter of the various claims was commonly owned at the time any inventions 
covered therein were made absent any evidence to the contrary. Applicant is advised of the obligation under 37 
CFR 1.56 to point out the inventor and invention dates of each claim that was not commonly owned at the time a later 
invention was made in order for the examiner to consider the applicability of 35 U.S.C. 103(c) and potential 35 
U.S.C. 102(e), (0 or (g) prior art under 35 U.S.C. 103(a). 



8. Claims 37, 39-45, 47, 49-54, and 55 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Smith et al. "Video Skimming and Characterization through the 
Combination of Image and Language Understanding Techniques", 17-19 June 1997, pp. 
775-781. 



As to claims 37 and 47: 

Smith teaches a data processing apparatus and method for processing media content 
comprised of a plurality of scenes (see Abstract), the apparatus comprising: 

• a plurality of segments each for describing one of the plurality of scenes of media 
content, the content description (e.g., selection video and audio portions ... This 
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segment contains 71 scenes, of which, skim A has captured 23 scenes, and skim B 
has captured 17 scenes; section 3.4) data further including: 

a context attribute having a value for describing a context of the media 
content (e.g., text in the video provides significant information as to the 
content of a scene; section 2.4), and 

a plurality of importance attributes each associated with one of the 
segments and having a value representing a degree of contextual 
importance of the corresponding one of the segments (e.g., identifying the 
most significant words in a given scene, and for image understanding, it 
entails segmentation of video into scenes, detection of objects of 
importance face and text 9 and identification of the structure motion of a 
scene; section 1.2); and 
• an output unit operable to output at least one of the segments based on at least 
one of the importance attributes (e.g., extract the significant audio and video 
information and create a "skim " video which represents a very short synopsis of 
the original; see Abstract /display only the video pertaining to a segment f s 
content ...It would show less video than the original and could be used to sample 
many segments without viewing each in its entirely. The amount of content 
displayed should be adjustable so the user can view as much or as little video as 
needed, from extremely compact to full-length video; section 1.1 & also see Fig. 
7)- 
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Smith does not specifically teach "input content description data". However, Smith 
suggests "an choosing the 'significant images and word' that should be included in the 
skim video" (section 1.2 & also see fig.3). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to include Smith's teaching for "input content description data" because it 
would have provided the capability for identifying the most significant words in a given 
scene as well as it entails segmentation of video into scenes and identification of the 
structure motion of a scene. 

As to claims 39 and 49: 

Smith teaches the content description data includes supplemental information (e.g., the 
amount of content displayed should be adjustable so the user can view as much or as 
little video as needed; see section 1.1). 

As to claims 40 and 50: 

Smith teaches the media content corresponds to video data and/or audio data (e.g., video 
information as well as audio information; see Abstract & section 1.1). 

As to claims 41 and 51: 

Smith teaches each of the plurality of segments is provided with linkage information for 
linking to dominant data that represents the segment (see Abstract). 
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As to claims 42 and 52: 

Smith teaches the dominant data is one or more of text data, image data and audio data 
(e.g., the audio portion; section LI & audio and images embedded within the video; 
section L2). 

As to claims 44 and 54: 

Smith teaches the context description data is previously generated outside of the data 
processing apparatus prior the inputting (see section 3.2). 

As to claims 43 and 53: 

4 

Smith teaches a plurality of context attribute and a plurality of importance attributes are 
associated with one segment (e.g., scenes, segments, and individual frames in video ... 
identifying the most significant words in a given scene, and for image understanding, it 
entails segmentation of video into scenes . . . objects of importance face and text ' and 
identification of the structure motion of scene; section 2). 

As to claims 45 and 55: 

Smith teaches the output unit is operable to output in response to a user query regarding 
the context (e.g., the relevant portions of video for the segments related to their query; 
section LI). 
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9. Claims 38, 46, 48, and 56- 59 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Smith et al. in view of B. Arons, "SpeechSkimmer: A System for Interactively 
Skimming Recorded Speech," ACM, March 1997, pp. 3-38. 

As to claims 38 and 48: 

Smith does not specifically teach "the plurality of segments are hierarchically 
described'' 

Arons teaches the plurality of segments are hierarchically described in the data structure 
portion (e.g., a hierarchy of segments can be created that roughly correspond to the 
spoken equivalents of sections, subsections, paragraphs, and sentences of a written 
document; section 2.3). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to combine the feature from Arons in the system of Smith because it would 
have provided the capability for locating important or emphasized portions of a 
recording, and selecting the equivalent of paragraph or new topic boundaries, for the 
purpose of creating audio overviews or outlines. 
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As to claims 46 and 56: 

Arons teaches the context description data further includes a plurality of time attributes 
each associated with a corresponding one of the segments for determining a start time and 
one of an end time and a duration of the scene associated with the corresponding segment 
(see sections 3.2 and 4.2.9). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to combine the feature from Arons in the system of Smith because it would 
have provided the capability for locating important or emphasized portions of a 
recording, and selecting the equivalent of paragraph or new topic boundaries, for the 
purpose of creating audio overviews or outlines. 

As to claim 57: 

The rejection of claim 37 above is incorporated herein in full. Additionally, Smith 
teaches: 

• the segment elements that are associated with the one of context attributes (e.g., 
regions from video frames thai contain textual information), and having a value 
representing a degree of importance of the scene associated with the one of the 
segment elements in relation to the context of the context attribute [section 2.4], 
and 

• selection means for selecting one or more of the segments based on an analysis of 
the importance attributes (e.g., choosing the "significant image and word" that 
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should be included in the skim video . .importance of each scene can be 
evaluated; section 1.2 & identifying significant objects that appear in the video 
frames is one of the key components for video characterization; section 2.4). 

Arons teaches: 

• the content description data being arranged in a hierarchy (e.g., a hierarchy of 
segments can be created that roughly correspond to the spoken equivalent of 
sections; subsections, paragraphs, and sentences of a written document; section 
2.3); 

• a plurality of segment elements each being a child of one of said section elements 
(see section 3.2) ; and 

• a plurality of time attributes each associated with a corresponding one of the 
segments for determining a start time and one of an end time and duration of the 
scene associated with the corresponding segment (see sections 3.2 and 4.2.9). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to combine the feature from Arons in the system of Smith because it would 
have provided the capability for locating important or emphasized portions of a 
recording, and selecting the equivalent of paragraph or new topic boundaries, for the 
purpose of creating audio overviews or outlines. 



Application/Control Number: 10/733,981 
Art Unit: 2176 



Page 13 



As to claims 58-59: 

They include the same limitations as in claims 44-45, respectively, and are similarly 
rejected under the same. 

Response to Arguments 

10. Applicant's arguments filed on 03/14/2006 have been fully considered but are deemed to 
be moot in view of the new grounds of rejection necessitated by Applicant's amendments. 

Conclusion 

11. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 



Arman et al. 


U.S. Pat. No. 5,606,655 


Issued: Feb. 25, 1997 


Yeo et al. 


U.S. Pat. No. 5,821,945 


Issued: Oct. 13, 1998 


Ratakonda 


U.S. Pat. No. 5,956,026 


Issued: Sep. 21, 1999 


Moriyama et al 


U.S. Pat. No. 6,732,109 


Issued: May 23, 2000 


Gu 


U.S. Pat. No. 6,075,875 


Issued: Jun. 13, 2000 


Tan et al. 


U.S. Pat. No. 6,356,902 


Issued: Mar. 12, 2002 


Christel et al., " 


Evolving Video Skim into Useful Multimedia Abstraction, 



Human Factors in Computing Systems, ACM 0111*98 Conference Proceedings, Los 
Angeles, pp. 171-178, April 1998. 
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MPEG-4 Overview - Atlantic City Version "International Organisation for 
Standardisation Coding of Moving Pictures and Audio" pp. 1-50, October 1998. 



12. Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1. 136(a) will be calculated from the mailing date of the 
advisory action. In no event, however, will the statutory period for reply expire later than 
SIX MONTHS from the mailing date of this final action. 

Contact information 

13. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Maikhanh Nguyen whose telephone number is (571) 272- 
4093. The examiner can normally be reached on Monday - Friday from 9:00am - 5:30 
pm. If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Heather Herndon can be reached at (571) 272-4136. 
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The fax phone number for the organization where this application or proceeding is 
assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published 
applications may be obtained from either Private PAIR or Public PAIR. Status 
information for unpublished applications is available through Private PAIR only. For 
more information about the PAIR system, see http://pair-direct.uspto.gov. Should you 
have questions on access to the Private PAIR system, contact the Electronic Business 
Center (EBC) at 866-217-9197 (toll-free). 
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WILLIAM BASHORfc 
PRIMARY EXAMINER 
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